Gauss on least-squares and maximum-likelihood estimation

Least squares and maximum likelihood

Estimation by Least Squares and by Maximum Likelihood

Although statements to the contrary are often made, application of the principle of least squares is not limited to situations in which p is normally distributed. The GaussMarkov theorem is to the effect that, among unbiased estimates which are linear functions of the observations, those yielded by least squares have minimum variance, and the independence of this property from any assumption re...

Maximum Likelihood Recursive Least Squares Estimation for Multivariable Systems

This paper discusses parameter estimation problems of the multivariable systems described by input–output difference equations. We decompose a multivariable system to several subsystems according to the number of the outputs. Based on the maximum likelihood principle, a maximum likelihood-based recursive least squares algorithm is derived to estimate the parameters of each subsystem. Finally, t...

Least squares methods in maximum likelihood problems

It is well known that the GaussNewton algorithm for solving nonlinear least squares problems is a special case of the scoring algorithm for maximizing log likelihoods. What has received less attention is that the computation of the current correction in the scoring algorithm in both its line search and trust region forms can be cast as a linear least squares problem. This is an important observ...

The extended least-squares and the joint maximum-a-posteriori maximum-likelihood estimation criteria

Approximate model equations often relate given measurements to unknown parameters whose estimate is sought. The Least-Squares (LS) estimation criterion assumes the measured data to be exact, and seeks parameters which minimize the model errors. Existing extensions of LS, such as the Total LS (TLS) and Constrained TLS (CTLS) take the opposite approach, namely assume the model equations to be exa...